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Motivated by novel results in the theory of complex adaptive systems, we analyze the dynamics 
of random walks in which the jumping probabilities are time- dependent. We determine the survival 
probability in the presence of an absorbing boundary. For an unbiased walk the survival probability 
is maximized in the case of large temporal oscillations in the jumping probabilities. On the other 
hand, a random walker who is drifted towards the absorbing boundary performs best with a constant 
jumping probability. We use the results to reveal the underlying dynamics responsible for the 
phenomenon of self-segregation and clustering observed in the evolutionary minority game. 



Random walk is one of the most ubiquitous concepts 
of statistical physics. In fact, it finds applications in 
virtually every area of physics (see e.g., [1-6] and ref- 
erences therein). Random walks in the presence of ab- 
sorbing traps are much studied in recent years as models 
for absorbing-state phase transitions [7,8], polymer ad- 
sorption [9], granular segregation [10], and the spreading 
of an epidemic [11]. 

In this Letter we analyze the problem of a random 
walk with an absorbing boundary, in which the jumping 
probabilities are time-dependent. In addition to the in- 
trinsic interest of such time- dependent random walks, our 
study is motivated by the flurry of activity in the field of 
complex adaptive systems. 

Agent-based models of complex adaptive systems are 
attracting significant interest across many disciplines 
[12]. They find applications in social, biological, physical, 
and economic sciences. Of particular interest are situa- 
tions in which members compete for a limited resource, or 
to be in a minority [13]. In financial markets for instance, 
more buyers than sellers implies higher prices, and it is 
therefore better for a trader to be in a minority group of 
sellers. Predators foraging for food will do better if they 
hunt in areas with fewer competitors. Rush-hour drivers, 
facing the choice between two alternative routes, wish to 
choose the route containing the minority of traffic [14]. 

One of the most studied models in the field of com- 
plex adaptive systems is the minority game (MG) [15], 
and its evolutionary version (EMG) [13] (see also [16-28] 
and references therein). The game describes agents who 
each make a binary decision (e.g., "to buy" /"to sell", or 
"taking route A" / "taking route B" ) at every point in the 
game. Profit is made by those who find themselves in the 
minority group, i.e. who end up selling when most wish 
to buy, or vice versa. Each winner gains R points (the 
"prize"), while agents belonging to the majority group 
lose 1 point (the "fine"). The agents have a common 
"memory" look-up table, containing the outcomes of re- 
cent occurrences. Faced with a given bit string of re- 
cent occurrences, each agent chooses the outcome in the 
memory with probability g, known as the agent's "gene" 
value. If an agent score falls below some value d, then its 



strategy (i.e., its gene value) is modified. In other words, 
each agent tries to learn from his past mistakes, and to 
adjust his strategy in order to survive. 

Early studies of the EMG were restricted to situations 
in which the prize-to-fine ratio R was assumed to be equal 
unity. A remarkable conclusion deduced from the EMG 
[13] is that a population of competing agents tends to self- 
segregate into opposing groups characterized by extreme 
behavior. It was realized that in order to flourish in such 
situations, an agent should behave in an extreme way 
(5 = or 5 = 1) [13]. 

On the other hand, in many real life situations the 
prize-to-fine ratio may take a variety of different values 
[25]. A different kind of strategy may be more favorite in 
such situations. In fact, we know from real life situations 
that extreme agents not always perform better than cau- 
tious ones. In particular, our daily experience indicates 
that in difficult situations (e.g., when the prize-to-fine 
ratio is low) human people tend to be confused and in- 
decisive. In such circumstances they usually seek to do 
the same (rather than the opposite) as the majority. 

Based on this qualitative expectation, we have recently 
extended the exploration of the EMG to include situa- 
tions in which the prize-to-fine ratio R differs from unity. 
It was found [25] that an intriguing phase transition ex- 
ist in the model: "confusion" and "indecisiveness" take 
over when the prize-to-fine ratio falls below some criti- 
cal value, in which case "cautious" agents (characterized 
by (7 = i) perform better than extreme ones. In such 
circumstances agents tend to cluster around g = ^ (see 
Fig. 1 of Ref. [25]) rather than self-segregate into two 
opposing groups. 

Moreover, it has been demonstrated [25,27] that the 
population of evolving agents never establishes a genuine 
stationary distribution. In fact, the probability of a par- 
ticular agent to win oscillates in time. This fact has been 
overlooked in former studies of the EMG. The score of 
each agent may therefore be described by a random walk 
with time- dependent jumping (winning) probabilities. 

Thus, the problem of random walkers whose jumping 
probabilities are time-dependent is of great interest for 
the understanding of the dynamics of complex adaptive 
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systems. The aim of the present Letter is to analyze this 
problem, and to provide an analytical explanation for the 
phase-transition observed in such systems. 

An unbiased random, walk. We study a discrete- 
time random walk on the nonnegative integers, Xt = 
0, 1, 2, . . . , with the origin absorbing. Initially the walker 
is at xq = 1 (We shall later generalize our results to 
include situations in which xq > 1-) The probability 
p{t) to step to the right is time-dependent and is given 
by p{t) = ^ — (—1)*^, where \A\ is the amplitude of 
the temporal oscillations in the jumping probabilities 

(-i<^<|)- 

We denote by y the location of the rightmost site yet 

visited. The probability P{x, y, t) for a random walker to 
be at a position x at time t follows the evolution equation 

P{x, y,t + 2) = [(i + Af + (i - Af]P{x, y, t) 

+{^-+A){^-A)[P{x-2,y,t) + P{x + 2,y,t)] , (1) 

for X < y ~ 2. We also have P{0,y,t) = 0, representing 
the absorbing boundary at the origin. Denoting D(jj, t) = 
P{y,y,t), the additional boundary conditions are 

D{y, t + 2) = (i - A^)[P{y - 2, y, t) + P{y-2,y- 1, t) 

+D{y-2,t)] + [\ + {-l)yAfD{y,t), (2) 



P[y-2,y,t + 2) = {-- A^) [P{y - 4, y, t) + D{y, t)] 



+2{- + A^)P{y-2,y,t), (3) 



and 



P{y-2,y-l,t + 2) = {-- A^)P{y -A,y-l,t) 

+2{-^+A^)P{y~2,y-l,t) 
+ [\-{-lYAfD{y-2,t). (4) 

It proofs useful to define the generating function 
P{x, y, z) = X;jlo z^P{x, y, t) (and similarly for D). Mul- 
tiplying Eqs. (l)-(4) by z* and summing over t, one finds 
that P{x,y,z) satisfies the recursion relation 



z-^Piy -2,y) = i-- A^)[P{y -4,y) + D{y)] 
+2{^+A^)P{y-2,y) , 



(7) 



and 



z-^P{y -2,y -!) = {-- A^)P{y - 4, y - 1) 

+2{\ + A^)P{y-2,y-l) 
+ [\-{-iyAfb{y-2). 



(8) 



The solution of Eq. (5) subject to the boundary con- 
dition at the origin, P{0,y) = 0, is P{x,y) = (A^ — 
X^^)B{y), where A is a function of z and A. To deter- 
mine the survival probability as t ^ oo, we shall analyze 
the behavior of P{x, y, z) as z ^ 1. In this limit 



C{x) = A^ - A" 



2 sinh 



(9) 



where e = 1 — z. 

Substituting Eqs. (7) and (8) in Eq. (6) one obtains 
the recursion relation 



(2 + ,)|^_(l + ,) 



C{y-^) 



B{y) = ^qB{y-l) 



+ l{l + q)S^Biy-2), (10) 
2 C{y - 4) 



whose asymptotic solution is 

B{y) 



sinh^(Viiy) i + {-l)yA 



— 1 



(11) 



where 17=2 + (^1)^^ and s = ^(L-a^) ' 

The survival probability is determined by S{t) = 
E^oELo-P(^'?-^'*)- Taking cognizance of Eqs. (9) 
and (11) one finds that the singular behavior of its gen- 
erating function, S{z), is dominated by 



S{z) = Y,B{y)Y.C{x) 



12{\ + A) 



(12) 



u=0 



x=0 



z-^P{x, y) = (^ - A^)[P{x - 2, y) + P{x + 2, y)] 
+2{\+ A^)P{x,y) , 



(5) 



(we drop the argument z for brevity), subject to the 
boundary conditions 



Nothing that for large t, the coefficient of z' in (1 — z)~i 
is l/\/7rf, we have that the asymptotic behavior of the 
survival probability is 



S{t;xo = 1) 



A 12 _ 
-t~^ . 



(13) 



z-^D{y) = {^-A^) [P{y -2,y) + P{y - 2, y - 1) 

+D{y-2)] + [\ + {-l)yAfb{y), (6) 



Finally, it is straightforward to prove the recursion re- 
lations S{t; xo ^2n+ 1, A) = {I + ^^)S'(t; xq = 1, A) 

and S{t;xo = 2n,A) = {^^)S{t;xo = I, -A). Thus, 
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the survival probability for an arbitrary value of the ini- 
tial position xq is given by 



S{t) 



xo + [i- {-ir"]A 



(14) 



We therefore conclude that an unbiased random walker 
whose jumping probabilities display large temporal oscil- 
lations (i.e., \A\ ~ i) has the largest survival probability. 
On the other hand, the survival probability S{t) has a 
minimum at A — for even values of xq (corresponding 
to a fixed, time-independent jumping probability), and 
at A = ^ for odd values of xq. 

Figure 1 displays the survival probability as a function 
of A, the amplitude of the temporal oscillations in the 
jumping probabilities. We find an excellent agreement 
between the analytically predicted results [see Eq. (14)] 
and the numerical ones. 




FIG. 1. The survival probability S{t;xo)y/^t'^ as a func- 
tion of the amplitude of the temporal oscillations in the jump- 
ing probabilities. We display results for different values of the 
initial location: xo = 1,^0 = 2, xq — 3, and xq = 4. Theoret- 
ical results are represented by solid curves. 

A biased random walk. We next consider situations 
in which the step length to the left is larger than the 
step length R to the right (where R < 1). Thus, the 
walker is drifted towards the absorbing boundary. We 
shall also generalize the analysis to include situations in 
which the jumping probability to the left is different from 
the corresponding jumping probability to the right. The 
(time- dependent) jumping probability to the right is now 



given by p(t) = ^ — e— (— 1)*A, where - 



-e < A < i 



In order to survive under such conditions, the walker 
must step to the right more times than he steps to the 
left. More precisely, at least a fraction / = of his 
steps must be to the right (at any given point in the 
walk). The chance for the mean number of right-steps to 
deviate from its average is exponentially decreasing with 



time. Using an analysis along the same lines as before, 
one finds that the asymptotic large t limit of the survival 
probability is now given by [29] 



S{t) = bt-^e 



-Ft 



(15) 



where the associated entropy function (or large-deviation 
function) reads 



2 



/+ln( ^) +(!-/+) In (i-^ 

p+J V 1 - -P+ 



+;_,„(t)+(i_/_„.,(i^ 

with p± = ^ ~ e ± A, and 



(16) 



1 - 46^ + AA^ 



- V(l - + AA^y - 64/(1 - f)A^ /8A . (17) 

[The explicit expression of the prefactor b — b{R) is not 
important for the analysis]. 

The survival probability at late times is dominated by 
the exponential factor e~^*. We note that e^^ [and thus 
also S{t)] is a monotonic decreasing function of \A\. One 
therefore concludes that a biased random walker (one 
who is drifted towards the absorbing boundary) with a 
constant jumping probability (i.e., with A — 0) has the 
largest survival probability. On the other hand, a (bi- 
ased) random walker with large temporal oscillations in 
his jumping probabilities has the poorest chances of sur- 
vival. 

Complex adaptive systems. We now apply the results 
derived in the present work to provide an explanation 
for the intriguing phase-transition observed in the EMG 
[transition from self-segregation to clustering (with an 
intermediate M-shaped phase), as the value of the prize- 
to-fine ratio R falls below some critical value] . 

As mentioned, it has been demonstrated [25] that the 
population of evolving agents never establishes a true sta- 
tionary distribution. In fact, the probability of a partic- 
ular agent to win (to step to the right in the terminology 
of this Letter) oscillates in time, the amplitude being de- 
pendent on the particular gene- value of the agent [27]. 
"Extreme" agents (with g — 0,1) have an oscillation am- 
plitude which is larger than the corresponding amplitude 
of "cautious" agents (those with g = i). In fact, the win- 
ning probability of g ~ i agents is practically constant 
in time. We therefore write A^ = a'^{R){g - i)^ [33]. 

One should also take into account the self-interaction 
(or so-called market impact in financial market terminol- 
ogy) that agents in such systems experience [18]. An 
agent has a smaller probability of winning when partic- 
ipating in the game as compared to an outsider, some- 
one whose action does not affect the outcome. The self- 
interaction term has the form £(5) = -^g^l — g) [18]. 
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Substituting the expressions for A{g) and £(17) in Eq. 
(16) one finds that [and thus also S{t)] has three 
quaUtatively different forms, depending on the precise 
value of R (the step-lengths ratio, or equivalently the 
prize-to- fine ratio). Figure 2 displays the function e~^, 
which determines the long-time survival probability of 
the agents. This figure demonstrates the phase-transition 
from self-segregation to clustering observed in the evolu- 
tionary minority game [13,25] (compare, in particular, 
with the numerical results presented in Fig. 1 of [25]). 
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FIG. 2. The survival probability for different values of the 
prize-to-fine ratio: R = 0.9983, R = 0.9991, and R = 0.9997. 
The parameters used in the figure are eo/VN = 10^'^ and 
a — 0.9. Compare this figure with Fig. 1 of [25]. The graphs 
are rescaled for convenience. 



There is also an intermediate phase [for Rc < R < 
i?i^^], in which the survival probability has an M-shaped 
distribution. This behavior is a direct consequence of 
two opposing factors: (a) the fact that a biased random 
walker with a constant jumping probability (A — 0, or 
equivalently g — has the largest survival probability, 
and (b) the fact that the market impact (which decreases 
the winning probability) is the largest ioi g = ^ agents. 
Note that the phase-transition from self-segregation to 
clustering becomes sharp as the number of agents N in- 
creases. In fact, the intermediate (M-shaped) phase dis- 
appears in the N 00 limit, in which case the transition 
occurs at = rP = 1 [see Eq. (18)]. 

In summary, in this Letter we have analyzed the dy- 
namics of random walks with time-dependent jumping 
probabilities. In particular, we have calculated the sur- 
vival probability of such walkers in the presence of an 
absorbing boundary. It was shown that the best strategy 
to be adopted by the walkers depends on the precise value 
of the step-lengths ratio R. In the unbiased case (i? = 1) 
the survival probability is maximized by a walker who 
has large temporal oscillations in his jumping probabili- 
ties. On the other hand, a random walker who is drifted 
towards the absorbing boundary is better off keeping his 
jumping probabilities constant (i.e., time-independent). 

Furthermore, we have shown that the results, when ap- 
plied to the theory of complex adaptive systems, provide 
a direct analytical explanation for the phase-transition 
(from self- segregation to clustering) observed in the evo- 
lutionary minority game. 
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For R > /Jc^-* the survival probability S{t) is peaked 
around 5 = and g — 1, and has a minimum at g = ^ . To 
survive under such conditions, an agent (a biased random 
walker) should behave in an extreme way (that is, should 
have large temporal oscillations in his jumping probabil- 
ities). On the other hand, for R < Rc [poor conditions, 
in which the fine (length step to the left) is larger than 
the reward (length step to the right)] one finds that S{t) 
is peaked around g — ^. This corresponds to cautious 
agents with constant (time-independent) jumping prob- 
abilities. Thus, cautious agents out-perform the extreme 
ones under harsh conditions. [It should be emphasized 
that this occurs despite the fact that the average winning 
probability of extreme agents (5) is actually larger than 
the corresponding probability of cautious agents (i — e)]. 



[1] M. N. Barber and B. W. Ninham, Random and Restricted 
Walks (Gordon and Breach, New York, 1970). 

[2] N. G. van Kampen, Stochastic Processes in Physics and 
Chemistry (North-Holland, Amsterdam, 1992). 

[3] R. Fernandez, J. Frohlich, and A. D. Sokal, Random 
Walks, Critical Phenomena, and Triviality in Quantum 
Field Theory (Springer Verlag, Berlin, 1992). 

[4] G. H. Weiss, Aspects and Applications of the Random 
Walk (North Holland, Amsterdam, 1994). 

[5] D. ben-Avraham and S. Havlin, Diffusion and Reactions 
in Fractals and Disordered Systems (Cambridge Univer- 
sity Press, Cambridge, 2000). 



4 



[6] R. Dickman and D. ben-Avraham, Phys. Rev. E. 64, 
020102(R) (2001). 

[7] J. Marro and R. Dickman, Nonequilibrium Phase Tran- 
sitions in Lattice Models, (Cambridge University Press, 
Cambridge, 1999). 

[8] H. Hinrichsen, Adv. Phys. 49, 815 (2000). 

[9] K. De'Bell and T. Lookman, Rev. Mod. Phys. 65, 87 
(1993). 

[10] Z. Farkasa and T. Fulop, J. Phys. A: Math. Gen. 34, 
3191 (2001). 

[11] P. Grassbergor, H. Chate, and G. Rousseau, Phys. Rev. 
E 55, 2488 (1997). 

[12] For a detailed account of previous work on agent- 
based models such as the Minority Game see 
http : // www /unifr .ch /econophysics. 

[13] N. F. Johnson, P. M. Hui, R. Jonson, and T. S. Lo, Phys. 
Rev. Lett. 82, 3360 (1999). 

[14] B. Huberman and R. Lukose, Science 277, 535 (1997). 

[15] D. Challet and C. Zhang, Physica A 246, 407 (1997); 
256, 514 (1998); 269, 30 (1999). 

[16] R. D'Hulst and G. J. Rodgers, Physica A 270, 514 
(1999). 

[17] E. Burgos and H Ceva, Physica 284A, 489 (2000). 

[18] T. S. Lo, P. M. Hui and N. F. Johnson, Phys. Rev. E 62, 

4393 (2000). 

[19] P. M. Hui, T. S. Lo, and N. F. Johnson, e-print cond- 
mat/0003309. 

[20] M. Hart, P. Jefferies, N. F. Johnson and P. M. Hui, e- 

print cond-mat/0003486; c-print cond-mat/0004063. 
[21] E. Burgos, H. Cova and R. P. J. Pcrazzo, c-print cond- 
rnat/0007010. 

[22] T. S. Lo, S. W. Lim, P. M. Hui and N. F. Johnson, Phys- 
ica 287A, 313 (2000). 

[23] Y. Li, A. VanDeemen and R. Savit, e-print 
nUn.AO/0002004. 

[24] R. Savit, R. Manuca and R. Riolo, Phys. Rev. Lett. 82, 
2203 (1999). 

[25] S. Hod and E. Nakar, Phys. Rev. Lett. 88, 238702 (2002). 

[26] A. C. C. Coolcn, c-print cond-mat/0205262. 

[27] E. Nakar and S. Hod e-print cond-mat/0206056; Phys. 
Rev. E (to be pubhshed) . 

[28] I. Caridi and H. Ceva, c-print cond-mat/0206515. 

[29] This result can also bo obtained using the Sparre An- 
dersen combinatorial relation [30,31]. See [32] for more 
details in the simpler case of time-independent jumping 
probabilities. 

[30] W. Feller, An Introduction to Probability Theory and its 
Applications, Volumes 1 & 2 (Wiley, New York, 1968, 
1971). 

[31] E. Sparre Andersen, Math. Scand. 1, 263 (1953); 2, 195 
(1954). 

[32] M. Bauer, C. Godreche and J. M. Luck, J. Stat. Phys. 

96, 963 (1999). 

[33] One can use a more general expansion of the form A^ = 
X]^iOfc(fl' ~ 1)^*1 but this would not change the final 
conclusions to be discussed below. 



5 



